VISION-LANGUAGE MODELS

A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning (2022.aacl-main)

Challenge:	Vision-language models are increasingly used to produce textual descriptions of visual content.
Approach:	They propose to disentangle description specificity from description length . they find people prefer more specific descriptions regardless of length based on their own subjective preferences .
Outcome:	The proposed model shows that people prefer more specific descriptions regardless of length.

Copied to clipboard

Hugo Berg, Siobhan Hall, Yash Bhalgat, Hannah Kirk, Aleksandar Shtedritski, Max Bain

Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs (2024.findings-naacl)

Challenge:	Large-scale, pretrained vision-language models are growing in popularity due to impressive performance on downstream tasks with minimal finetuning.
Approach:	They propose to apply ranking metrics to image-text representations to investigate bias measures and debiasing methods to reduce various bias measures.
Outcome:	The proposed model reduces bias measures with minimal degradation to image-text representations.

Copied to clipboard

Victor Carbune, Hassan Mansoor, Fangyu Liu, Rahul Aralikatte, Gilles Baechler, Jindong Chen, Abhanshu Sharma

A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models (2024.findings-emnlp)

Challenge:	Visual language models (VLMs) are achieving increasingly strong performance on multimodal tasks.
Approach:	They propose to transfer reasoning capabilities from large-language models to VLMs by constructing a 20x larger dataset and a larger dataset to improve general reasoning capabilities.
Outcome:	The proposed model outperforms larger models without an upstream OCR system while keeping inference time constant.

Copied to clipboard

Ashutosh Sathe, Prachi Jain, Sunayana Sitaram

Semantic Text Processing

Challenge:	Existing studies have highlighted the existence of social biases within large vision and language models.
Approach:	They propose a framework for systematically evaluating gender, race, and age biases in vision-language models with respect to professions.
Outcome:	The proposed framework covers all supported inference modes of the recent vision-language models, including image-to-text, text-to image, and image- to-image.

Towards Difficulty-Agnostic Efficient Transfer Learning for Vision-Language Models (2024.emnlp-main)

Copied to clipboard

Yongjin Yang, Jongwoo Ko, Se-Young Yun

MTIVE: Multi-Task Image Verification Engine Using Vision-Language Models for E-commerce (2026.acl-industry)

Challenge:	Vision-language models (VLMs) have demonstrated remarkable applicability across downstream tasks, including zero-shot image classification.
Approach:	They propose an efficient transfer learning method that integrates visual prompts and text adapters with pre-trained VLMs to achieve optimal performance for any target domain.
Outcome:	The proposed method outperforms baselines on unseen tasks.

Copied to clipboard

Yu-Tong Cao, Vishnu Prabhakaran, Arunita Das, Purav Aggarwal, Anoop Saladi

Controlling Multimodal Conversational Agents with Coverage-Enhanced Latent Actions (2026.acl-long)

Challenge:	Vision-language models struggle with noisy real-world images and multi-task requirements.
Approach:	They propose a curriculum learning framework that adapts vision-language models through three stages . MTIVE uses frozen base weights with stacked LoRA adapters for shared domain knowledge .
Outcome:	MTIVE outperforms open-source and proprietary baselines in standard and continual learning settings.

Copied to clipboard

Yongqi Li, Hao Lang, Tieyun Qian, Yongbin Li

Semantic Text Processing

Challenge:	Recent reinforcement learning (RL) has been widely explored for adapting MCAs to various human-AI interaction scenarios.
Approach:	They propose to use a latent action space for reinforcement learning instead of RL to fine-tune MCAs.
Outcome:	The proposed method outperforms baselines on two conversation tasks with a novel cycle consistency loss.

Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning (2025.findings-emnlp)

Copied to clipboard

Yihong Tang, Ao Qu, Zhaokai Wang, Dingyi Zhuang, Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, Jinhua Zhao

TURTLEAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics (2026.findings-acl)

Challenge:	Currently, vision-language models excel in many downstream tasks but struggle with spatial reasoning, which is crucial for navigation and interaction with physical environments.
Approach:	They propose a framework that generates synthetic data to provide targeted supervision for VLMs across these basic spatial capabilities.
Outcome:	The proposed framework disentangles 2D spatial reasoning into three core components: direction comprehension, distance estimation, and localization.

Copied to clipboard

Chao Wen, Jacqueline Staub, Adish Singla

SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data (2025.acl-long)

Challenge:	Vision-language models have been explored for visual programming, but performance is unclear . most prior work focuses on visual programming for productivity .
Approach:	They propose a visual programming benchmark that uses visual programming to evaluate VLMs.
Outcome:	The proposed model improves on GPT-5, GPT-4o, and Qwen2-VL-72B on real-world tasks by 20% . the proposed model is based on 823 visual programming tasks in the Turtle Graphics domain .

Copied to clipboard

Michael Ogezi, Freda Shi

GeoCoder: Solving Geometry Problems by Generating Modular Code through Vision-Language Models (2025.findings-naacl)

Challenge:	Vision-language models struggle with spatial reasoning, a skill that humans excel at.
Approach:	They propose to use a spatial-reasoning Enhanced (SpaRE) VLM to improve spatial reasoning in visual question answering and robotics.
Outcome:	The proposed model achieves a 49% performance gain on the What's Up benchmark while maintaining strong results on general tasks.

Copied to clipboard

Aditya Sharma, Aman Dalmia, Mehran Kazemi, Amal Zouaq, Christopher Pal

VIPHY: Probing “Visible” Physical Commonsense Knowledge (2023.findings-emnlp)

Challenge:	Various vision-language models (VLMs) have made significant progress in multimodal tasks, but they still struggle with geometry problems.
Approach:	They propose a vision-language model that leverages modular code-finetuning to generate and execute code using a predefined geometry function library.
Outcome:	The proposed model improves geometric reasoning abilities by 16% on a GeomVerse dataset compared to other methods.

Copied to clipboard

Shikhar Singh, Ehsan Qasemi, Muhao Chen

Reasoning

Challenge:	Existing studies have demonstrated that vision-language models can retain and generalize knowledge, but they do not measure their ability to retain it.
Approach:	They build an automatic pipeline to derive a knowledge resource for calibrating and probing vision-language models.
Outcome:	The proposed model outperforms the pretrained model on size and spatial tasks.

MAPWise: Evaluating Vision-Language Models for Advanced Map Queries (2025.naacl-long)

Copied to clipboard

Srija Mukhopadhyay, Abhishek Rajgaria, Prerana Khatiwada, Manish Shrivastava, Dan Roth, Vivek Gupta

Natural Language Interfaces

Challenge:	Vision-language models excel at tasks requiring joint understanding of visual information and natural language.
Approach:	They propose to use choropleth maps to answer questions from three geographical regions in the United States, India, China as question templates.
Outcome:	The proposed model outperforms other models in the area of visual language and visual question answering.

Aligning VLM Assistants with Personalized Situated Cognition (2025.acl-long)

Copied to clipboard

Yongqi Li, Shen Zhou, Xiaohu Li, Xin Miao, Jintao Wen, Mayi Xu, Jianhao Chen, Birong Pan, Hankun Kang, Yuanyuan Zhu, Ming Zhong, Tieyun Qian

Self-play through Computational Runtimes improves Chart Reasoning (2025.findings-acl)

Challenge:	Existing studies on vision-language models aligned with general human objectives have not been successful because people with diversified backgrounds have different cognition even in the same situation.
Approach:	They propose to characterize individuals based on the sociological concept of Role-Set and then evaluate their actions to see whether personalized alignment is achieved.
Outcome:	The proposed framework constructs a cognition-aware and action-based reward model for personalized alignment.

Copied to clipboard

Tautvydas Misiūnas, Hassan Mansoor, Jasper Uijlings, Oriana Riva, Victor Carbune

VPL: Visual Proxy Learning Framework for Zero-Shot Medical Image Diagnosis (2024.findings-emnlp)

Challenge:	Vision-language models (VLMs) achieve impressive zero-shot performance on multimodal reasoning tasks.
Approach:	They propose a self-play programming interface which leverages visual reasoning models to generate code to decompose a complex visual reasoning task in sub-tasks and use it as a tool to solve decomposed tasks.
Outcome:	The proposed approach outperforms baselines on difficult chart reasoning benchmarks.

Copied to clipboard

Jiaxiang Liu, Tianxiang Hu, Huimin Xiong, Jiawei Du, Yang Feng, Jian Wu, Joey Zhou, Zuozhu Liu

From Mimesis to Metamorphosis: Evolving VLM Judges via In-Context Comparing and Knowledge Internalization (2026.findings-acl)

Challenge:	Insufficient medical text precision and the modal disparity between text and vision spaces pose challenges for vision-language models like CLIP.
Approach:	They propose a visual proxy learning framework that combines a text refinement module and a stable Sinkhorn algorithm to enhance the diagnostic performance.
Outcome:	The proposed model outperforms the state-of-the-art CLIP inference by 1.69% to 15.31% on five datasets covering various diseases.

Copied to clipboard

Juntuo Wang, Yuming Qiao, Yifan Yang, Lunxi Yuan, Liang Luo, Dan Meng

Information Retrieval

Challenge:	Existing approaches to subjective assessment are inconsistent and inconsistent due to inconsistent scales and inherent preference biases.
Approach:	They propose a framework that operationalizes subjective assessment as comparative analysis and internalizes it via Language Buttons.
Outcome:	The proposed framework achieves state-of-the-art performance across multiple benchmarks and is scale-steerable.

When Seeing Overrides Knowing: Disentangling Knowledge Conflicts in Vision-Language Models (2026.acl-long)

Copied to clipboard

Francesco Ortu, Zhijing Jin, Diego Doimo, Alberto Cazzaniga

Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games (2026.findings-acl)

Challenge:	Vision-language models combine visual and textual information to perform complex tasks. conflicts between internal knowledge and external visual input can lead to hallucinations and unreliable predictions.
Approach:	They propose to use a dataset to identify attention heads that deliberately contradict internal commonsense knowledge to resolve cross-modal conflicts.
Outcome:	The proposed model can be manipulated to find out which visual inputs are conflicting . the model can then be orientated towards internal parametric knowledge or visual information .

Copied to clipboard

Keyang Zhong, Junlin Xie, Hefeng Wu, Haofeng Li, Guanbin Li

Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective (2024.emnlp-main)

Challenge:	Vision-language models have shown impressive capabilities in perceptual tasks . however, they degrade in complex multi-hop reasoning under multi-player game settings .
Approach:	They propose a multi-agent framework for evaluating and synthesizing role-driven game scripts . they use curated and synthetic datasets to model uncertainty and deception .
Outcome:	The proposed model significantly boosts the performance of VLMs in narrative reasoning and hidden fact extraction under uncertain, adversarial, and socially complex conditions.

Copied to clipboard

Zhaotian Weng, Zijun Gao, Jerone Andrews, Jieyu Zhao

Med-VRAgent: A Framework for Medical Visual Reasoning-Enhanced Agents (2025.emnlp-main)

Challenge:	Current methods to learn biases from the perspective of model components are limited by their complexity and performance.
Approach:	They propose a framework that incorporates causal mediation analysis to measure and map the pathways of bias generation and propagation within vision-language and multimodal tasks.
Outcome:	The proposed framework is applicable to a wide range of vision-language and multimodal tasks and reduces bias by 22.03% and 9.04% in the MSCOCO and PASCAL-SENTENCE datasets.

Copied to clipboard

Guangfu Guo, Xiaoqian Lu, Yue Feng

MAFMO: Multi-modal Adaptive Fusion with Meta-template Optimization for Vision-Language Models (2025.findings-emnlp)

Challenge:	Visual Language Models (VLMs) have shown strong performance in tasks like radiology report generation but struggle with hallucinations, vague descriptions, Inconsistent logic and poor localization.
Approach:	They propose a framework for medical visual reasoning based on Visual Guidance and Self-Reward paradigms and Monte Carlo Tree Search to improve the model's visual reasoning capabilities.
Outcome:	The proposed framework outperforms existing models on multiple medical VQA benchmarks.

Copied to clipboard

Mingrui Xie, Lulu Xu, Junliang Du

Medical Vision-Language Pre-Training for Brain Abnormalities (2024.lrec-main)

Challenge:	Existing approaches focus on single-modality adjustments, leading to suboptimal alignment and limited generalization.
Approach:	They propose a plug-and-play framework for visual recognition that integrates a Harmonic Cross-Modal Adapter and a Meta-Template Optimization module.
Outcome:	Extensive experiments across multiple fine-grained visual recognition benchmarks show that MAFMO consistently improves existing methods’ performance on both novel classes and harmonic mean while maintaining robustness under various challenging conditions with minimal computational overhead.

Copied to clipboard

Masoud Monajatipoor, Zi-Yi Dou, Aichi Chien, Nanyun Peng, Kai-Wei Chang

From Off-Policy to On-Policy: Enhancing GUI Agents via Bi-level Expert-to-Policy Assimilation (2026.acl-long)

Challenge:	Existing vision-language models lack expertise for medical applications due to the scarcity and complexity of data.
Approach:	They propose a pipeline to collect medical image-text aligned data for pretraining from public resources such as PubMed and build a high-performance vision-language model tailored to specific medical tasks.
Outcome:	The proposed model is based on a large brain image-text dataset and will be released to the public.

Copied to clipboard

Zezhou Wang, Ziyun Zhang, Xiaoyi Zhang, Zhuzhong Qian, Yan Lu

Challenge:	Vision-language models are increasingly deployed as computer-use agents that operate desktops and browsers.
Approach:	They propose a method that turns static expert traces into policy-aligned guidance . they propose RLVR with a per-task, dynamically updated cache to decompose planning and execution .
Outcome:	The proposed model improves UITARS1.5-7B success from 22.87% to 32.13% on OSWorld-Verified and raises a held-out split from 5.74% to 10.30% on MMBench-GUI and Online-Mind2Web.

Can VLMs Actually See and Read? A Survey on Modality Collapse in Vision-Language Models (2025.findings-acl)

Copied to clipboard

Mong Yuan Sim, Wei Emma Zhang, Xiang Dai, Biaoyan Fang

VLA-Mark: A cross modal watermark for large vision-language alignment models (2025.emnlp-main)

Challenge:	Vision-language models integrate textual and visual information, enabling them to process visual inputs and generate predictions.
Approach:	They review work on modality collapse analysis to provide insights into the reason for this unintended behavior and review probing studies for fine-grained vision-language understanding.
Outcome:	The proposed models can achieve competitive performance in vision-language tasks despite relying heavily on textual information and ignoring visual information.

Copied to clipboard

Shuliang Liu, Zheng Qi, Jesse Jiaxi Xu, Yibo Yan, Junyan Zhang, He Geng, Aiwei Liu, Peijie Jiang, Jia Liu, Yik-Cheung Tam, Xuming Hu

Robustness in NLP

Challenge:	Existing text watermarking methods disrupt visual-textual alignment, leaving semantic-critical concepts vulnerable.
Approach:	They propose a vision-aligned framework that embeds detectable watermarks into outputs . they combine localized patch affinity, global semantic coherence, contextual attention patterns .
Outcome:	The proposed framework shows lower PPL and higher BLEU than conventional methods with near-perfect detection (98.8% AUC).

Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chains (2025.acl-long)

Copied to clipboard

Juntian Zhang, Chuanqi Cheng, Yuhan Liu, Wei Liu, Jian Luan, Rui Yan

Mixed Signals: Decoding VLMs’ Reasoning and Underlying Bias in Vision-Language Conflict (2025.findings-emnlp)

Challenge:	Existing vision-language models struggle to disentangle information scattered across complex visual inputs, leading to performance degradation.
Approach:	They propose a focus-centric visual chain paradigm that enhances VLMs’ perception, comprehension, and reasoning abilities in multi-image scenarios.
Outcome:	The proposed approach achieves average performance gains of 3.16% and 2.24% across two distinct model architectures, without compromising the general vision-language capabilities.

Copied to clipboard

Pouya Pezeshkpour, Moin Aminnaseri, Estevam Hruschka

SemVink: Advancing VLMs’ Semantic Understanding of Optical Illusions via Visual Global Thinking (2025.emnlp-main)

Challenge:	Vision-language models have demonstrated impressive performance by effectively integrating visual and textual information to solve complex tasks.
Approach:	They build upon existing benchmarks to create five datasets containing mismatched image-text pairs and examine how they reason over visual and textual data .
Outcome:	The proposed model reasoned over visual and textual data in real-world applications but not in the visual and visual descriptions.

Copied to clipboard

Sifan Li, Yujun Cai, Yiwei Wang

Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models (2025.acl-long)

Challenge:	Vision-language models excel in semantic tasks but fail at detecting hidden content . current architectures prioritize abstract reasoning over low-level visual operations .
Approach:	They propose a benchmark to test vision-language models that can detect hidden content . they propose HC-Bench to scale images to low resolutions to unlock 99% accuracy .
Outcome:	HC-Bench shows that leading VLMs achieve near-zero accuracy even with explicit prompting . et al.: current models prioritize abstract reasoning over low-level visual operations . they urge a shift toward hybrid models bridging gap between computational vision and human cognition .

Copied to clipboard

Ido Cohen, Daniela Gottesman, Mor Geva, Raja Giryes

Making VLMs More Robot-Friendly: Self-Critical Distillation of Low-Level Procedural Reasoning (2025.emnlp-main)

Challenge:	Vision-language models excel at extracting and reasoning about information from images, yet their capacity to leverage internal knowledge about specific entities remains underexplored.
Approach:	They propose a dataset which allows separating entity recognition and question answering . they hypothesize that this decline arises from limitations in how information flows from image tokens to query tokens.
Outcome:	The proposed model performance drops when the entity is presented visually rather than textually.

Copied to clipboard

Chan Young Park, Jillian Fisher, Marius Memmel, Dipika Khullar, Seoho Yun, Abhishek Gupta, Yejin Choi

See or Say Graphs: Agent-Driven Scalable Graph Understanding with Vision-Language Models (2026.findings-acl)

Challenge:	Large language models (LLMs) are human-centric, but omit low-level, spatially grounded details needed for robotic execution.
Approach:	They propose a lightweight framework for vision-language procedural planning that enables iteratively critique, revise and verify their own plans without external supervision or teacher models.
Outcome:	a new framework outperforms weaker models 100X the size in vision-language procedural planning . the framework generates higher-quality, execution-ready plans that can be used at inference and fine-tuning .

Copied to clipboard

Shuo Han, Yukun Cao, Zezhong Ding, Zengyi Gao, S Kevin Zhou, Xike Xie

SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis (2026.findings-acl)

Challenge:	Existing studies have explored textual graph descriptions and visual modalities for VLMs to understand graphs.
Approach:	They propose a unified framework that enhances both scalability and modality coordination in graph understanding by integrating textual and visual modalities.
Outcome:	GraphVista scales to large graphs, 200 larger than those used in existing benchmarks, and consistently outperforms existing textual, visual, and fusion-based methods.

Copied to clipboard

Zijian Wu, Jinjie Ni, Xiangyan Liu, Zichen Liu, Hang Yan, Michael Qizhe Shieh